Using Kohonen's Self-Organizing Feature Map to Uncover Automobile Bodily Injury Claims Fraud

Claims fraud is an increasingly vexing problem confronting the insurance industry. In this empirical study, we apply Kohonen's Self-Organizing Feature Map to classify automobile bodily injury (BI) claims by the degree of fraud suspicion. Feed forward neural networks and a back propagation algorithm are used to investigate the validity of the Feature Map approach. Comparative experiments illustrate the potential usefulness of the proposed methodology. We show that this technique performs better than both an insurance adjuster's fraud assessment and an insurance investigator's fraud assessment with respect to consistency and reliability. INTRODUCTION AND BACKGROUND One vexing problem confronting the property-casualty insurance industry is claims fraud. Individuals and conspiratorial rings of claimants and providers unfortunately can and do manipulate the claim processing system for their own undeserved benefit (Derrig and Ostaszewski, 1994; Cummins and Tennyson, 1992). The National Insurance Crime Bureau (NICB) estimates that the annual cost of the insurance fraud problem is $20 billion, which is equivalent to the cost of a Hurricane Andrew each year (NICB, 1994). According to the National Health Care Association, insurance fraud in health insurance represented an estimated 10 percent surcharge on the U.S. $550 billion annual health care bill in 1988 (Garcia, 1989). A recent Insurance Research Council report on automobile insurance fraud stated that “the excess injury payments as a result of fraud and/or buildup are estimated to be between 17 and 20 percent of total paid losses, or $5.2 to $6.3 Patrick Brockett is the Gus S. Wortham Memorial Chairholder in Risk Management and Professor of Finance, Mathematics, and Management Science and Information Systems at the University of Texas at Austin. Xiaohua Xia is Vice President, Research and Risk Management at AutoBond Acceptance Corporation in Austin, Texas. Richard Derrig is Senior Vice President of the Automobile Insurers Bureau of Massachusetts and Vice President, Research for the Insurance Fraud Bureau of Massachusetts. The authors wish to thank the Automobile Insurers Bureau of Massachusetts for providing the data and computing support for this research. This article was reviewed and accepted by the previous editor. The Journal of Risk and Insurance 246 billion additional for all injury claims in 1995.” (IRC, 1996) Outside of the United States, fraud claims are also increasing. For example, arson was thought to be costing the United Kingdom £500 million a year in 1991 (Wilmot, 1991). Private passenger automobile bodily injury (BI) insurance is the largest line of property-casualty insurance in the U.S. It is estimated that about 40-50 percent of BI claims, for Massachusetts at least, contain some degree of suspicion concerning fraud (Derrig, Weisberg and Chen, 1994). The proportion of fraudulent claims also appears to be increasing as evidenced by ever higher rates of bodily injury claims per accident. The Insurance Research Council documents a countrywide change from 22 BI claims per property damage liability claim (the proxy for claims per accident) in 1987 to 29 BI claims per accident in 1992 (IRC, 1994). While the entire increase may not be due to an increase in fraudulent BI claims, the increase is indicative of fraudulent or abusive insurance lotteries (Cummins and Tennyson, 1992). With the awareness of the increasing frequency of suspected claims fraud, more and more rigorous techniques and empirical databases are being created for the purpose of fraud detection. One such database is the National Insurance Crime Bureau (NICB) Database System, which contains 200 million records of claims and stolen vehicles and which was recently made available to member insurance companies (Dauer, 1993). Once a company enters a claim in the database, either the NICB or that company's special investigation unit (SIU) will commence an investigation if some suspicious information arises for that particular claim. In Massachusetts, a detail claim database (DCD) of all auto BI claims has been assembled commencing January 1, 1994. This database, available to company special investigative units (SIUs) and the Insurance Fraud Bureau, is expected to provide detailed information on approximately two hundred thousand claims annually. In addition to databases, people began to use other approaches to analyze the automobile bodily injury (BI) claims fraud problem. Using statistical methods, Weisberg and Derrig (1991) determined the mechanisms behind automobile BI claims fraud, such as relationships between injury type and treatment, for example. Their studies of 1985/1986 and 1989 BI claims found that the overall level of suspected or apparent fraud was about 10 percent of the claims, while the apparent build-up level was 35 percent in 1985/86 and 48 percent in 1989. Nearly all companies rely on the training of personnel as the primary method of recognizing claims suspected of fraud. Specified objective and subjective claim characteristics, such as “no witnesses to the accident,” have become known as potential fraud indicators or red flags. Three-quarters of the companies rely on the presence of these red flags to assist the claim adjuster in recognizing suspicious The Insurance Research Council reported that about half the property and casualty premium volume was written by companies with special investigative units (SIUs) (IRC, 1984). Given the recent increased emphasis on fraud detection, the establishment of SIUs has expanded greatly. In Massachusetts, for example, all companies writing private passenger auto as a servicing carrier must establish an SIU. 2 The term "build-up" is defined as an attempt on the part of the claimant and/or the health provider to inflate the damages for which compensation is being sought (Derrig and Ostaszewski, 1994). Predicting Bodily Injury Claims Fraud 247 claims, and one-quarter of those companies use automated methods of tracking red flags (IRC, 1992). Studies have also shown that the insurance industry does not share a consensus definition regarding what constitutes claims fraud. Weisberg and Derrig (1993) found that different BI claims handling professionals had ambiguous perceptions of what constitutes BI claims fraud. For example, in a coding of the same set of claims by two sets of adjusters, each set of adjusters classified approximately 9 percent of claims as apparently fraudulent, but ironically only 1.8 percent of the identified claims were simultaneously considered to be apparently fraudulent by both sets of adjusters. Derrig and Ostaszewski (1994) further verified the lack of concordance of the fraud perceptions among different BI handling professionals, such as between insurance company claims adjusters and insurance investigators. In order to study the problem Derrig and Ostaszewski (1994) applied a fuzzy setbased clustering technique. The results of the study again showed the lack of concordance among people with respect to which claims were fraudulent. Based upon their findings, the analysts concluded that the use of an adjuster's judgment, as compared to that of an investigator, can serve well in first-pass screening of BI claims regarding suspicion levels. Weisberg and Derrig (1993) used regression models to discern which objective and subjective fraud indicators are more significant than others in effectively identifying suspicion levels of BI claims fraud. If the goal is to identify individual fraudulent claims then their studies exposed several problems. For example, they used only the adjuster's and investigator's subjective assessments of BI claims fraud as dependent variables. As noted above, however, these dependent variables were not consistent with each other, and there was apparent ambiguity and overlap between them. Another problem was that the reliability of each dependent variable couldn’t be verified in the real world, due to data limitations. Additionally, statistical approaches, including regression methods used by Weisberg and Derrig (1993), have difficulty handling the 65 binary fraud indicators as independent variables unless the sample size is sufficiently large. Thus, based upon correlation analysis, some 25 indicators were chosen as the independent variables in the regression models; the other 40 indicators were not utilized for practical reasons. Due to these limitations, Derrig and Ostaszewski (1993) did not use the fraud indicators of claims to extract characteristics of fraudulent claims and construct a screening device directly. Rather, they used fuzzy clustering of multiple suspicion levels pertaining to the accident, the claimant, the insured, the treatment, the injury and the lost wages. 3 Unlike other fraud detection problems, such as credit card fraud, most automobile bodily injury claims cannot ultimately be verified. It is either too costly or impossible to determine and classify without doubt a fraudulent BI claim unless a reliable court decision is available. However, insurance companies tend not to resolve a claim in this manner because it is both risky and costly. As a result, the data used by Derrig, Weisberg and Ostaszewski, and in this study, consists of only objective and subjective indicators or subjective assessments and are not based upon the legal conclusions of whether or not legal fraud was probably present with respect to the BI suspected claims fraud. 4 For logistical and expense considerations, real company claim operations may be more inclined to track from 10 to 25 indicators systematically rather than 65 indicators. Hence, parsimonious solutions may have more practical value. The Journal of Risk and Insurance 248 Weisberg and Derrig (1991) claimed that at that time it was premature to address the ultimate goals of quantifying the amount of fraud and developing guidelines for detecting and controlling fraud. Since then analysts studying the automobile BI claims fraud problem have been working to ultimately develop a BI claims fraud detection system or claim classification system. Besides the work done by the Automobile Insurance Bureau of Massachusetts (Weisberg and Derrig, 1993; Weisberg and Derrig, 1996), there have been other attempts in this direction. For instance, Artis, Ayuso and Quillen (1997) model the behavioral characteristics of the claimants and insureds in the Spanish automobile insurance market. An expert system has been developed in Canada “to aid insurance company adjusters in their decision making and to ensure that they are better equipped to fight fraud” (Belhadji and Dionne, 1997). In this empirical study, we intended to apply a different approach to build a BI claim fraud detection or classification system. Specifically, we apply a neural network approach, Self-Organizing Feature Maps (Kohonen, 1982, 1989, and 1990), to construct a claim classification system that uses similar collections of fraud indicators as the classifier. In the second stage of the study, we do a comparative study between the feature map BI claim classification system and both the adjuster’s assessment and the investigator’s assessment. Claim adjusters and investigators represent the two primary forces in claim processing and fraud detection. Their expertise will serve as a good benchmark for a novel quantitative approach such as the feature map method. The tool used in the comparative study is another neural network model. Specifically, a feed-forward neural network model combined with a backpropagation learning algorithm. Particularly, we would like to see whether the feature map approach can perform better than the adjuster's and investigator's subjective assessments as measured by the consistency achieved in assessing suspicion levels and clustering BI claims An overview of the paper is as follows: following this background information, the second section describes the empirical BI claims data used in the study. This data sample is used to construct the feature map models and apply them to BI fraud detection problem in the next two sections. Feed-forward neural network models are constructed to test the feature map approach in the penultimate section. A summary section concludes the paper. The Kohonen’s Self-Organizing Feature Map Algorithm is presented in an Appendix. BI CLAIMS FRAUD DATA The data set was generated in a study of Massachusetts BI claims and previously was analyzed by Weisberg and Derrig (1991,1992, and 1993). The entire BI claim data set consists of 127 claims, selected from among 387 claims for accidents in 1989. The data production process was completed in two steps. In a first pass through all 387 claims data, each claim was independently examined by two claim adjusters. Of these, 62 claims were deemed to be apparently fraudulent by at least one of the adjusters. The other 65 claims out of the total 127 claims were randomly sampled from the remaining 325 apparently non-fraudulent claims. In a second Predicting Bodily Injury Claims Fraud 249 pass through the data, these 127 claims were again independently coded, this time by an insurance adjuster and by an investigator from the Insurance Fraud Bureau of Massachusetts. Each claim in the data set consists of a claim ID number, a vector of fraud indicators (we will use claim vector, pattern, and pattern vector interchangeably) and two professional assessments of the suspicion level of fraud, i.e., the adjuster's assessment, and the investigator's assessment. In total, there are 65 fraud indicators which have been divided into six categories based upon the practice used in automobile insurance claim processing: characteristics of the accident, the claimant, the insured, the injury, the treatment and the lost wages. Some indicators, such as the accident characteristics, were based on the police report and witness testimony, while others were collected from actual claim files. Every indicator is a dummy variable and assumes a binary value based upon the answer to a yes-no question. The adjuster’s and investigator’s assessments, which reflect their opinions of the level of suspicion for the claim, fall into a range between 0 and 10, with 10 standing for a virtually certain fraudulent claim, and 0 for a virtually certain valid claim. The distribution of their assessments is shown in Table 1. Adopting the convention used by Weisberg and Derrig (1992), we transform the ten-point suspicion level variable into four discrete categories: not suspicious (0), slightly suspicious (1-3), moderately suspicious (4-6) and strongly suspicious (7-10). Table 1 Breakdowns by Adjuster Assessment and Investigator Assessment. Adjuster Assessment Training Set Holdout Set Combined Suspicion Count % Count % Count % 0 35 45% 19 38% 54 43% 1-3 11 14% 16 32% 27 21% 4-6 12 16% 10 20% 22 17% 7-10 19 25% 5 10% 24 19% Total 77 100% 50 100% 127 100% Investigator Assessment Training Set Holdout Set Combined Suspicion Count % Count % Count % 0 32 42% 19 38% 51 40% 1-3 5 6% 6 12% 11 9% 4-6 11 14% 10 20% 21 17% 7-10 29 38% 15 30% 44 35% Total 77 100% 50 100% 127 100% It is necessary to emphasize the fact that, besides the subjective assessments of two professionals, there are no so-called observed (true) fraudulent levels available in the data set. This makes statistical prediction methods, such as logistic 5 It is normally true since an insurance claim is rarely identified in practice as a fraudulent claim unless some extensive special investigation and/or legal procedures are involved. The Journal of Risk and Insurance 250 regression, impossible to implement in the first place unless we are willing to assume the validity of the professionals’ subjective assessments. Not surprisingly, building a reliable dependent variable to measure the level of suspicion is one of the major goals of this study. We realized that neural network models are normally very computationally intensive. Kohonen Feature Maps, as described in an Appendix, are no exception. Based upon our experience, training a feature map until it reaches a stable state often takes hours on clustered Unix stations. It often becomes too time and resource consuming to conduct extensive experiments based upon advanced sampling techniques. In our first experiment, we thus decided not to do multiple samples or use other advanced sampling techniques. All 127 claim vectors were randomly mixed together before 77, or about 60 percent of the claims, were randomly chosen as the training data set, with the remaining 50 claims used as the holdout data set. We selected a 60:40 proportion for the purpose of having as many claims in the training set as possible but reserved “enough” claims to test the validity of the methodology. The distributions of the professionals’ assessments based upon this first sampling in terms of the level of suspicion are summarized in Table 1. We can easily see that the percentage of valid claims perceived by the adjuster and the investigator are comparable. However, among the same 127 BI claims the investigator found more claims to be “strongly suspicious” than did the adjuster. For example, the investigator found 10 more claims (29 versus 19) “strongly suspicious” claims than did the adjuster in the training data set alone. In our second experiment, we took 9 additional random samples from our data set of 127 claims. The size of the training samples was still 77 claims, or 60 percent of the 127 claims. These 9 random samples plus the original sample were run on smaller feature maps (smaller number of output units). With the multiple samplings, the common results are expected to be more robust. APPLYING KOHONEN FEATURE MAPS It must be assumed that if two claims have common or similar fraud indicator patterns, the result would be an approximately equivalent level of suspicion. Consequently, two claims whose indicator vectors have a sufficiently short distance between them should be assigned similar suspicion level values. Hence, if there is a mapping from the claim patterns (vectors) to a suspicion level assessment, such that similar assignments go to the similar patterns and different assignments to different patterns, then we can claim such a mapping system is consistent and reliable. This is the consistency or continuity principle of unsupervised pattern recognition approaches (Schalkoff, 1992). Cluster Analysis has been a popular method dealing with unsupervised learning problems (lack of observed values of output variables). Statistical software packages such as SAS include various cluster analysis models. Certain optimization criteria are applied to cluster analyses to split a set of observations into a number of groups or a hierarchical structure. The distribution of observations among groups is determined by the optimization criteria. Hence, it Predicting Bodily Injury Claims Fraud 251 might happen that one of the groups has a single or very few vectors while another group has 99 percent of the vectors. The application of optimization rules in cluster analysis leads to a lack of control over the numerical process that might have better performance if the process were interactive and certain a priori knowledge were to be incorporated. In cases where the data sample is small but its dimensionality is high, or simply the data is quite noisy, conventional cluster analyses might leave little of the necessary freedom for professionals to provide expert inputs into the decision making process. Another disadvantage associated with conventional cluster analyses is that it is not easy to identify the groups in terms of the nature of the observations in each group, such as which group represents the “strongly suspicious” insurance claims and which group represents the apparently “valid” claims in this particular case. Kohonen’s Feature Maps (Kohonen, 1982, 1989, and 1990) are capable of dealing with this unsupervised problem while overcoming the weaknesses inherent in conventional cluster analyses. See Appendix for the description of a typical feature map algorithm. For the demonstration, we used an 18×18 square feature map and trained it for 2,000 epochs. Each cell in the 18×18 map is assigned a random “weight vector” of the same dimension as the claim or pattern vector – namely the number (65 in this case) of fraud indicator variables. The training data set is the one from our first sampling and consists of 77 claims and their accompaning pattern vectors. We know that a distance can be calculated between each cell’s weight vector and a pattern vector. For any given pattern vector, we first found the maximum distance among all the weight vectors to the pattern vector and then subtracted each distance from the maximum. The result became a measure of the “closeness” between the weight vectors and the pattern. We then depicted the “closeness” in a threedimension space as shown in Figures 1 through 4. In the graphs, the height of the landscapes measures the “closeness” of the weight vectors to the pattern vector, i.e., the higher the landscape at location (i, j), the better the matching between the pattern vector and the weight vector (or the output unit) at (i, j). Figure 1 shows the landscape for a claim pattern pA after 2,000 epochs of learning, and adjusting the cells’ weight vectors in the manner to be specified subsequently. It is clear that the highest peak lies in the upper corner, which implies that this claim pattern finds its best-matching output unit in that corner. Notice the clear pattern of the landscape decreasing towards the lower corner, this means that the matching between pattern pA and the output units (their weight vectors) gets worse and worse towards the lower corner. It seems from this landscape that the training for the demo feature map was perfectly done. But the next figure shows the presence of noise. Figure 2 is the landscape for another claim pattern, pB. There are two things clearly different in this landscape. First, the highest peak is located at the lower corner. This implies that claim pattern pB might be quite different than the claim pA. Second, besides the highest peak, there are other noticeable peaks, i.e., the landscape does not present a near perfect pattern like the one in Figure 1. This suggests that the training might not have been perfectly executed, or that the training data set contains enough noise to make the identity of this claim ambiguous. The Journal of Risk and Insurance 252